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Abstract 


In this paper we construct a Rankin-Selberg integral which represents the Spinio x 
St L-function attached to the group GSOiqxPGL 2 . We use this integral representation 
to give some equivalent conditions for a generic cuspidal representation on GSOiq to 
be a functorial lift from the group G 2 x PGL 2 . 

1 Introduction 

In this paper we construct a new Rankin-Selberg integral which represents an L func¬ 
tion corresponding to the group GSOiq x PGL 2 . More precisely, let vr denote a 
generic cuspidal representation defined on the group GSOiq{A), and let r denote a 
cuspidal representation defined on PGL 2 {A). For simplicity, we shall assume that vr 
has a trivial central character. The L group of GSOiq{A) x PGL 2 {A) is the group 
G'S'pinio(C) X SL 2 {C). Consider the 32 dimensional irreducible representation of this 
group given by SpiniQ x St. Here SpiniQ is the 16 dimensional irreducible Spin rep¬ 
resentation of GS'pmio(C) and St denotes the standard representation of the group 
SL 2 {C). To this irreducible representation, one can attach the 32 degree partial L 
function denoted by {SpiniQ x St,Tr x r, s). 

To study this L function we construct a global integral which is given by 



( 1 ) 


Z(A)G50io(F)\GSOio(A) 


Here is a vector in the space of the representation vr, Z is the center of GSOiq 
and E{g,s) is an Eisenstein series which described in the beginning of section 2. The 
interesting representation in integral o is the representation Or- This representation 
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is constructed as a residue of an Eisenstein series defined on the group GSOio{A) as 
described fully in section 3. The main two properties of this representation are first, 
its dependence on the cuspidal representation r, and second, its smallness. In fact, as 
we prove in section 3, this representation is attached to the unipotent orbit (3^1). 

After showing that this integral is Eulerian, and that we obtain the above L func¬ 
tion, we then study the poles of this L function. We show that it can have at most a 
simple pole at s = 1. It is well known, see that the stabilizer of a generic point 
in the space of the representation S'pinio(C) x SL 2 {C), is the group G 2 (C) x SL 2 {C). 
Hence one expects that the above L function will have a simple pole at s = 1 if and 
only if the cuspidal representation vr is the functorial lift from G 2 x PGL 2 . In fact 
this is exactly what we prove. Indeed, the main result of this paper is given by the 
following theorem, which is stated and proved in section 6. (See section 6 for precise 
notations). 

Main Theorem: Let vr be an irreducible generic cuspidal representation of the group 
GSOiq{A) which has a trivial central character. Then the following are equivalent: 

1) The partial L function L^{Spinio x St, tt x r, 2s — 1/2) has a simple pole at s = 3/4. 

2) The period integral 

TiT{g)OT{g)0{g)dg ( 2 ) 

SOio(F)\SOio{A) 

is nonzero for some choice of data. 

3) There exists a generic cuspidal representation a of the exceptional group G' 2 (A) such 
that TT is the weak lift from the representation a x t of the group G 2 (A) x PGL 2 (A). 

We now describe the content of the paper. In section two we introduce the global 
integral we consider, and show that it is Eulerian. We try to do it in an abstract way. 
Indeed, we assume the existence of a representation 9 defined on the group GSOio(A) 
which is attached to the unipotent orbit (3^1). Using that data, we prove in theorem 
1, that the global integral is Eulerian. This method has the advantage that it will work 
for any representation which will satisfy the properties listed in the theorem. This 
means, that for any such representation the integral will be Eulerian. In section 3 
we construct an example of such a representation, which we denote by 0^, by means 
of a residue of an Eisenstein series. We are well aware of the existence of other such 
representations as well. 

Section 4 is devoted to the unramified computations. The local integral and L- 
function are each expressed as a power series in y, and where x is the 

character from which r is induced. The coefficients are traces of irreducible represen¬ 
tations of Spinio(C). The power series are simplified by multiplying them by a certain 
polynomial. The desired equality is equivalent to a formula for tensor products of 
representations of “rectangular shape” which is due to Okada. 

Sections 5 and 6 are devoted to the main theorem. The idea of the proof is as 
follows. The hard part in the theorem is to prove that part 2 implies part 3. To do 
that we first show that the cuspidal representation vr is an endoscopic lifting from a 
cuspidal representation e x r defined on the group Spq{A) x S'L2(A). This part is not 
trivial. To do it we need to use a new construction of a lifting which was announced in 
|(I2| and is a work in progress in |(I3j . In section 5 we give the precise details we need. 
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We don’t give all the proofs, they will appear in |(13j . but we give enough details for 
the reader to get the whole picture. Then, assuming part two of the above theorem, 
we show that the representation e defined on Spq{A) is actually a functorial lift from 
a generic representation a dehned on the exceptional group G2(A). 

Finally, we wish to remark that the representation 9^ seems to occur in other 
constructions of Rankin-Selberg integrals. Recently, we constructed some other global 
integrals, using this representation, which we proved to be Eulerian. It is our full 
intention to try to find the L functions they represent. 

2 A Global Integral 

Let G denote the similitude orthogonal group GSOiq. Let tt denote a generic irreducible 
cuspidal representation on the group G{A). We shall assume that it has a trivial central 
character. Let P denote the standard maximal parabolic subgroup of G whose levi part 
is GLi X GL^ which contains the standard Borel subgroup consisting of upper unipotent 
matrices. We shall denote by U{P) the unipotent radical subgroup of P. Let E{g,s) 
denote the Eisenstein series defined on the group G{A) which is associated with the 
induced representation 

Let 0 denote any automorphic representation defined on the group G{A). Assume 
that it has a trivial central character. Eor a cusp form in the space of tt we consider 
the integral 

Pn{g)0{g)E{g,s)dg (3) 

Z(A)G(F)\G{A) 

Here Z is the center of the group G. We are mainly interested in understanding 
what conditions we need to impose on the representation 9 so that integral © will be 
Eulerian with the Whittaker function defined on the representation tt. 

In terms of matrices we consider the group G relative to the form defined by the 
matrix Jiq. Here and elsewhere, the matrix is the n x n matrix with ones on the 
other diagonal and zeros elsewhere. For 1 < i < 5, let ai denote the five simple roots of 
the group G. Let Xq. (r) denote the one dimensional unipotent subgroup corresponding 
to the root ai. We label the roots such that 

Xai (r) = / + re ^2 Xa 2 (r) = / + re 2^3 Xa^ (r) = / + re 3^4 
Xa^ (r) = I + re' 4 ^^^ Xag (r) = / + re 4 6 

Here I is the 10 x 10 identity matrix and eG = Cij — eii_jpi_j. For 1 < z < 5 let 
w\i] denote the simple reflection corresponding to the simple root Oj. We shall write 
w\iii 2 .. .ir] for w[ii]w[i 2 ]. ■ ■ w\ir\. 

Let V’ denote a nontrivial character on the group E\A. Let U denote the maximal 
unipotent subgroup of G which consists of upper triangular matrices. We define two 
characters on the group U. For an automorphic form (j) dehned on G(A) we dehne its 
Whittaker model as 

W^{g) = J (t){ug)'ilTu~^{u)du 

U(F)\UiA) 
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Here, for u = (rtij) G U, we define ip[j{u) = + '“ 2 , 3 + 1 ^ 3 , 4 + '^ 4 , 5 + ?r 4 , 6 )- Similarly, 

we define 

= J (l){ug)il)u{u)du 

UiF)\U(A) 

where now tpuiu) = ' 0 (^* 1,2 + ^* 2,3 + ^^ 4,5 + ^^ 4 , 6 )- 

In KI-R-Si| it is explained how to associate with a unipotent class of a classical 
group a set of Fourier coefficients. We also use the notation Oq as explained there. 
Roughly speaking, if a is an automorphic representation of the group G one defines 
OG{a) as follows. It is defined to be the set of all unipotent classes of G such that for 
all O' with the property that if O' is greater or not related to a member in Og{o'), then 
a has no nontrivial Fourier coefficient corresponding to the unipotent class O' . Also, 
the representation a has a nonzero Fourier coefficient corresponding to any unipotent 
class O in OGic). 

We are now ready to prove the following 

Theorem 1: We keep the above notations. Let 9 he an automorphic representation 
of the group G{A) which satisfies: 

1) Og{9) = (SH). 

2) The integral is not zero for some choice of data. 

Then, integral is Eulerian and for Re{s) large it equals 

J J W^{g)9^’'l’^{g)fs{w[53]xa3{ri)xa3+as,{r2)g)dridr2dg ( 4 ) 

Z{A)U{A)\G{A) A2 

Proof: For Re{s) large we unfold integral Q and we obtain 


Tn{g)9{g)fs{g)dg 


(5) 


Z{A)P{F)\G{A) 


Next we expand 6 (g) along the unipotent radical of P. The Levi part of P acts on 
U{P) with three orbits. Indeed, we may identify U{P) with all matrices in G of the 

form ^ . Thus we may parameterize the orbits by the rank of the matrices X, 

which must be even. It follows from the cuspidality of tt that the rank zero and rank 
two orbits contribute zero to the integral. Denote 


eUiPU(g) = 


X{F)\X{A) 


X 

I 


V’(X2,1 + X2,,2)dX. 


The stabilizer of this character inside the Levi part of P is the group GSp^. Thus © 
equals 

j Tnig)9^^^^''^{g)fsig)dg (6) 

Z(A)GSpiiF)UiP){F)\GiA) 

Consider the unipotent group L which is generated by the matrices Iio + re'i 2 ; Lio + 
re'i^; Jio + Rq + re'^ 5 . We expand along the group L{F)\L{A). 
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The group GSp^ acts on L with two orbits. By the cuspidality of vr, the trivial orbit 
contributes zero to the integral. Denote V = L ■ U{P), and define a character ipv of V 

as follows. For v ^ V write v = lu with I = (lij) G L and u = ^ G U{P). We 

define 'ipv{v) = + 2 : 2,1 + 2 : 3 , 2 )- Thus we are left with the open orbit and hence 

integral equals 




(7) 


Z(A)R(F)V{A)\G{A) 


where 


e^Pv[g) = J 9{vg)^pviv)di 


V{F)\V{A) 


and similarly we define (g)- Also, the group R = GL 2 ■ Y where 


^ — {( 7 io + ?/ie 2 , 3 )( 7 io — ?/ie 4 ^ 5 )(/io + y 2 e 2 , 4 )( 7 io + ?/ 2 e 3 , 5 )( 7 io + 2/362,5)} 


The group GL 2 is embedded in G as all matrices of the form diag{\h\, \h\, h, 1, \h\, h, 1,1) 
where h G GL 2 and \h\ = deth. 

Next we consider the Fourier expansion 


eypv{g) = 




F\A 


-i&F' 


'f\a 


We claim that each summand in the summation over a on the right hand side is zero. 
Indeed, since a 7 ^ 0 it follows that the integration over ^3 together with the integration 
over V, produces a Fourier coefficient which corresponds to the unipotent class (52^1). 
By assumption 1) in the theorem it follows that these Fourier coefficients are all zero. 
Thus we are left only with the constant term. 

Denote w = u;[35]. Let Yc = Ro + 2/3625 ^'^d denote Vi = wVYcW~^. To describe 
the group Vi in term of matrices, let Ui denote the standard unipotent radical of the 
maximal parabolic subgroup of G whose Levi part is GLi x GSOg. We consider the 
character il^jj as a character of the group Ui by restriction. Next we consider the 
unipotent subgroup of U defined by all matrices of the form 


l{ri,r2, 2-3, ^4, rs, re, rr) = Ro + rie2^3 + r2e2^4 + rse^^g + r4e2,6 + r5e2j + r6e2_8 + rre^t^g 

Finally, we consider the group of all unipotent matrices of the form z{mi,m 2 ) = 
Rq + 771164 3 + 277260 3. A matrix multiplication implies that we have the factorization 
vi = uil(ri,0,r3,0,r5,rQ,r7)z(mi,m2) where vi G Vi and ui G Ui. We also consider 
the character il^jj as a character of the group Vi by restriction. Thus, for the above 
factorization, we have ipuivi) = ipuiuRipRi + r-j). From all this we obtain that (O 
equals 

{wg)fs{g)dg (8) 

Z{A)GL2(F)Y(F)Yc{A)V{A)\G(A) 


5 


We proceed with the following Fourier expansion 

gVi,tpuf 0, r4,0,0,0)rcg')'!/;((5ir2 + (52r4)(ir2dr4 

Collapsing summation with integration this also equals 

J (^2:{mi,rn2)wg)drnidrn2 

A2 

Here V2 is the unipotent subgroup of U generated by the group Ui and by the unipotent 
group consisting of all matrices /(ri, r2, ra, r4, rs, rg, ry). We view 'i/jgr as a character of 
V2 by restriction. Performing the same expansion for (p^^, integral (jSl) equals 

J J {z{m3,mi)wg)e^^’^^{z{mi,m2)wg)fs{g)drnidg ( 9 ) 

A4 


where the g integration is as in 0 . Next we expand along the unipotent 

subgroup of U generated by all matrices of the form t(ri,r2,rs,r4) = ho + ^634 + 
^263 5 + ^363 g + r4e3 7. It is not hard to check that the only nonzero contribution to the 
expansion comes from the constant term. Indeed, all other terms in the expansion will 
produce Fourier coefficients which correspond to unipotent classes which are greater 
than ( 3 ^ 1 ). By assumption 1 ) these Fourier coefficients are zero. Let V3 denote the 
unipotent subgroup of U generated by 1^2 and all matrices of the form t{ri,r2,ro,ri). 
The above discussion implies that = 9 V3,iPu ^ Factoring the integration over the 

group Y, integral © equals 

// / (t(0,r2,0,r4)z(m3,m4)wg)d^^’'^^(z(mi,m2)wg)fs(g)dridmjdg 

A4 (F\A)2 

( 10 ) 

Here the g variable is integrated over Z(A)GL2(T)y(A)I/(A)\G(A). 

We now expand the function in the above integral along the matrices 

t(ri,0,r3,0) with points in F\A. The group GL2 acts on this expansion with two 
orbits. The trivial one contributes zero to the integral because of the cuspidality of vr. 
Thus (nn equals 


J J {z{m3,rn4)wg)e^^’^^{z{rni,rn2)wg)fs{g)dmjdg 

Z(A)GLi{F)N'{F)Y{A)V{A)\G{A) A^ 

( 11 ) 

Here N' is the maximal unipotent subgroup of GL2 which, when embedded inside G, is 
a subgroup of U. The group GLi consists of all diagonal matrices inside the group G of 
the form diag{a, a, a, 1 , 1 , a, a, 1 , 1 , 1 ). The character V’(/, which was defined before the 
theorem, is viewed as a character of V3 by restriction. Next we factor the integration 
in m over the group N. The group GLi acts on this group with two orbits. The 
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trivial one contributes zero by cuspidality. Thus, m equals 

J j WT,{z{m^,mi)wg)e^''*’^{z{mi,m2)wg)fs{g)dmjdg ( 12 ) 

Z{A)Uo{A)\G{A) A4 

Here Uq is the group generated by V and N. Next we conjugate the matrix z{m3,m4) 
across w and collapse summation with integration. We also change variables g i—> w~^g. 
Thus integral m equals 


W^{g)e’^’'^^{z{rni,rn2)wg)fs{w[53]g)drnjdg 


( 13 ) 


Z{A)Ui{A)\G{A) A2 


where Ui is the subgroup of U defined as follows. Let Uq be the subgroup of Uq where 
we omit the one dimensional unipotent subgroups corresponding to the roots as and 
«3 Tees- Then Ui = w~^UqW. Factoring the integration Ui{A)\U{A), we obtain 
integral 0. ■ 


3 A Construction of a Small Representation 

In this section we construct a representation of the group G(A) and show that it 
satisfies the assumptions stated in Theorem 1 . This representation will depend on a 
choice of a cuspidal representation of GL2{A). 

Let T denote an irreducible cuspidal representation of the group GL2{A). We 
will assume that it has a trivial central character. Let (t{t) denote the symmetric 
square lift of r to GL^{A). This lift was constructed by Gelbart and Jacquet in uni- 
Let ^(r) = T ® T denote the tensor product representation of GS04{A). Denote by 
Q the standard maximal parabolic subgroup of G whose Levi part is GL^ x GSO4. 
We shall denote by U{Q) its unipotent radical. Let ET-{g,s) denote the Eisenstein 
series defined on the group G'(A) which is associated to the induced representation 

From the Langlands theory the poles of this Eisenstein series are determined by 
the poles of the constant terms. Since we induce from cuspidal data, it follows that 
we only need to consider the constant term along U{Q). This is easily computed and 
hence the poles of Er{g, s) are determined by 

L^{a{T) X /u(t), 6s — 3 )L‘^((T(r), 12 s — 6) 

L^{(t{t) X /u(t), 6s — 2)L'S'((t(t), 12s — 5) 

where 5 is a finite set of places, including the archimedean ones, such that outside of 
S all data is unramified. The above L function is equal to 

L^{a{T) X iT(r), 6 s — 3 )L'^(iT(r), 6 s — 3)L'^((t(t), 12s — 6 ) 

L^{a{T) X iT(r), 6 s — 2 )L’^(cr(r), 6 s — 2)L^{a{T), 12 s — 5) 

from which we deduce that the Eisenstein series has a simple pole at s = 2 / 3 . We 
denote Or{g) = Resg^2/s). In the rest of this section we will show that this 
representation does satisfy the assumptions of Theorem 1 . 
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Since (t(t) and /u(t) are both generic, the integral which was defined at 

the beginning of section 2 , is not zero for some choice of data. Since this integration 
corresponds to a unipotent orbit of the type (3^1), all we need to verify is that 9r 
has no nonzero Fourier coefficient which corresponds to any unipotent class which is 
greater than or not related to (3^1). 

To do that we use the same method as in section two. We start by 

studying the unramified local representation corresponding to Or- Let F be a local 
nonarchimedean field where r is unramified. Assume that r is a constituent of the 
induced representation , where y is an unramified character of F* and 

B 2 is the Borel subgroup of GL 2 . By the definition of the symmetric square lift, the 
unramified local representation corresponding to cr(r) is ^3 is the 

Borel subgroup of GL 3 , and xi is defined as xiidia,g{a,b, c)) = x^(ac“^). In a similar 
way we have ^{t) = Ind^^'^‘^X 2 d^B 4 ■ -^4 is the Borel subgroup of GS '04 and 

X 2 {diag{abr,ar,a~^,a~^b~^)) = x^(a 6 )y(r). Here the factor xif) occurs because we 
assume that ^(r) has a trivial central character. 

From all this we deduce 

Lemma 2: Let Q'r denote the unramihed constituent of Or at a nonarchimedean place. 
Then 0'^ is a sub-quotient of Ind^xs^Q^ ■ Here, for all g € GL 3 and h G GSO 4 we 
define Xsiig^d)) = x?{detg)^{X{h)) where X{h) denotes the similitude factor of the 
matrix h. 

Proof: Let B denote the Borel subgroup of G. The unramihed representation O'r is a 
constituent of the induced representation Ind^Xid^s'^ where X 4 (t) = { 0 - 10 ^ 3 ^ o,i)x{^) 

and t = diag{rai,ra 2 ,ra 3 , ra^, r, 1, 03 ^, 0 ^^, a^^). Let wq denote the Weyl element 

of G which has a one at the entries 

(1,1); (2,4); (3, 8 ); (4, 2); (5, 6 ); ( 6 , 5); (7, 9); ( 8 , 3); (9, 7); (10,10) 

and zero elsewhere. Then Ind^Xid^s'^ is isomorphic to Ln(i^X 4 °'^i/^ where X 4 °(^) = 
X 4 (^a’o This can be written as Ind^^Xbd^BiX&d^Bi)^]^"^ where X 5 (i) = X^(aia 2 a 3 ) 

and Xeit) = From this it follows that is a sub-quotient of IndpXsdQ^- ■ 

We return to the global situation. We need to prove that Or has no nonzero Fourier 
coefficient which corresponds to any unipotent class which is greater than or not related 
to (3^1). As mentioned in section two, in |(I-H,-S 1 j it is explained how to associate with 
a unipotent class of a group G a set of Fourier coefficients. It is clear that Or is not 
generic. Hence it has no nonzero Fourier coefficient with respect to the unipotent 
class (91). Arguing as in |(T-B,-Slj lemma 2.6 we deduce that if Or has a nonzero 
Fourier coefficient corresponding to the unipotent class (73) then it has a nonzero 
Fourier coefficient which corresponds to the unipotent class (71^). Similarly, if Or 
has a nonzero Fourier coefficient which corresponds to the unipotent class (5ri ... r^) 
then it has a nonzero Fourier coefficient which corresponds to the unipotent class 
(51®). Arguing as in |(T-B,-S2j lemma 3, we deduce that if Or has no nonzero Fourier 
coefficients which corresponds to the unipotent class (51®) then it has no nonzero 
Fourier coefficient corresponding to the unipotent class (71^). We should remark that 
these last statements are true for any automorphic representation of G and not only 
for Or- Hence it is enough to show that Or has no nonzero Fourier coefficients which 










correspond to the unipotent classes (51^) and to (4^1^). We now describe the families 
of Fourier coefficients which correspond to these two unipotent classes. 

Let V denote the unipotent radical subgroup of the standard parabolic subgroup 
of G whose Levi part is GLf x GSOe- We view 14 as a subgroup of the maximal 
unipotent subgroup 17 of G which consists of upper triangular matrices. Let a G 
F*. We define a character ■i/'v.a of the group V as follows. For v = (vij) € V, set 
i^v,a{v) = + ^ 2,5 + o,V 2 fi)- Thus the Fourier coefficient of the representation Or 

which corresponds to the unipotent class (51®) is given by 

j dTivg)'ilJvA'>^)dv 

V(F)\ViA) 

Next we describe the Fourier coefficient which corresponds to the unipotent class (4^1^). 
To do that, let R denote the subgroup of U defined as the group generated by all one 
dimensional unipotent subgroup associated with the positive roots of G where we omit 
the roots 01 , 03,04 and 05 . Thus, the dimension of R is 16. For r = (vij) € R we 
define '4^R{r) = + r 2,4 + The Fourier coefficient which corresponds to the 

unipotent class (4^1^) is given by 


J 9r{rg)il;R{r)dr 
R[F)\R(A) 

To prove our result we will show that the local unramified component O'r of Or does 
not support a local functional of the above type. More precisely, we prove 
Lemma 3: Let O denote one of the unipotent orbits (51®) or (4^1^). Then Or has 
no nonzero linear functional lo,a which satisfies lo,a{pA)A — 4 ’va(''^)^ 0 ,a{x) and no 
nonzero linear functional Iq which satisfies loipAx) = A^oix)■ This is for all 
V G V, r £ R and x is a vector in the space of Or- Here we denoted by p the action of 
the representation Or ■ 

Proof: Arguing as in |G-R.-S2j section 2 lemma 2 we have to show that the repre¬ 
sentation IndqXsdQ^ does not support any of these functionals. From the Bruhat 

theory this reduces to the problem of showing that IndQXsdq'^ has no admissible dou¬ 
ble coset in the space Q\G/L where L is either V or R. By that we mean that for 
any g G Q\G/L there is I € L such that glg~^ G Q and that V’l(0 7 ^ 1- Here i/jr 
is either or i/jr. From the Bruhat decomposition we obtain that each element 
g G Q\G/L can be written as g = wu where w is a Weyl element of G and u G U. 
As in |G-R,-S2j section 2 lemma 2 we deduce that wu is not admissible if and only if 
w is not admissible. Let U{Q)~ denote the transpose group of U{Q). Thus U{Q)~ 
consists of lower unipotent matrices. Let xi(ri) = Iio -|- rie'i 2 -,X 2 {r 2 ) = 7io -|- ^262 5 
and X 3 (r 3 ) = Iio -|- These are precisely the three one dimensional unipotent 

groups on which the character -0y,a is not trivial. To prove that w is not admissi¬ 
ble it is enough to show that for some i, we have wxi{ri)w~^ G Q. Assume not. 
This means that wxi{ri)w~^ G U(Q)~ for all i. Since tc is in G we have that 
if Wij = 1 then wn-iyi-j = 1. This means that if wx 2 {r 2 )w~^ = Iio -|- ^’ 2 e(j 
then wx^{r^)w~^ = ho + rAn-ij- Hence if wx 2 {r 2 )w~^ G \U{Q)~,U{Q)~] then 
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wx 3 {r 3 )w~^ G Q. Similarly, if wx^{r^)w~^ G [U{Q)~, U{Q)~] then wx 2 {r 2 )w~^ G Q. 
From this it follows that both wx 2 {r 2 )w~^^wx 2 ,{r^)w~^ G U'(Q)~/[U'(Q)~,17(Q)~]. 
Since a:i(ri) does not commute with both X 2 (x 2 ) and x^lr^), it follows the same for 
wxi{ri)w~^. Hence, also wxi{ri)w~^ G U{Q)~/\U{Q)~,U{Q)~]. However, there is no 
one dimensional unipotent subgroup of G which corresponds to a root in G, which has 
the property that it is in U{Q)~/\U{Q)~,U{Q)~] and which will commute with both 
wx 2 {r 2 )w~^ and wx 3 {r 3 )w~^. Thus every w is not admissible and the lemma is proved 
for the functional lo,a- A similar proof holds for the group R and the functional Iq. ■ 

4 The Unramified Computation 

In this section we carry out the unramified computation of the local integral which 
corresponds to integral Q. Let F denote a nonarchimedean field, and assume that all 
groups are defined over F. Let tt denote an unramified irreducible generic representa¬ 
tion of the group G. We assume that it has a trivial central character. We denote by 
I{s) the local induced representation Indf^dp. Let r denote an unramified irreducible 
representation of GL 2 and assume that it has a trivial central character. As in section 
3 lemma 2 we denote by 6'^ the local unramified constituent of the global representation 
Ot- The integral we consider is 

/ / WT,{g)e'^'^'^{g)fs{w[h2^^Xai{ri)xai+a5{r2)g)'4’{ri)dridr2dg (14) 

ZU\GF^ 

Assume that all functions are the K fixed vectors in their space of representation, 
where K is the maximal compact subgroup of G. Let SpiniQ denote the 16 dimensional 
irreducible Spin representation of GS'pinio(C), the L group of G. The corresponding L 
function was defined in m We denote by L{Spinio x St, n x t,s) the 32 dimensional 
L function which consists of the tensor product of these two representations. In this 
section we prove 

Proposition 4: For all unramiEed data, and for Re{s) large, integral (1141) equals 

L(Spinio X St, TT X T, 2s - 1/2) 

C(8s)C(8s - 2) ^ ^ 

Proof: Let T denote the maximal torus of G. Using the Iwasawa decomposition, 
integral m equals 

f W^{t)e'^’'^^{t)d]/^{t) I fsiw[^3,]xa3{ri)xa3+a5{r2)t)'i/{ri)dridr2dt 
Z\T 

where B is the Borel subgroup of G. We parameterize an element t in Z\T as 
t = diag{abi,ab 2 ,ab 3 ,ab 4 ,a,l,b'^^,b 2 ,b/ ). For this parameterization 6p{t) = 
\a\-^yi\-^\b2\-^\b3\-^\b4\-^. 

Next we compute the inner integration along the two dimensional unipotent sub¬ 
group. This is done exactly as in mi section 3 right after identity (11). We thus 
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obtain 

j fs{w[53]xa3{ri)xa3+a5{r2)t)'ipiri)dridr2 = 

Here we used the fact that 6p{t) = |a|^'^®| 6 i 6263 & 4 |^^. Also, we denote q = \p\ where p 
is a generator of the maximal ideal of the ring of integers in the field F. 

Recall, from section 3, that is the local component of a residue of an Eisen- 

stein series at the point 2/3. Hence we have 




% 






/ abi 


\ 


btJ 


Here Wr is the unramified Whittaker function corresponding to the induced represen¬ 
tation Ind'^^^xib ^3 of GL^ which was defined in section 3. Similarly, the function 
WrxT is the unramified Whittaker function corresponding to the induced representa¬ 
tion ■ Denote KT^{t) = {t) and also 



(bi \ 


/ abi 

\ 

Kr 

b2 

K by 

Krxr 

V 

a 

1 

b-y) 


Wr 



/ 064 


b2 


Wr 




\ 

|6/^63||o|"D2|54|-i 



Using the fact that 



ap 16162 ^ 3 P we obtain that integral (|T11) equals 


C(8s-2) f 

C( 8 s) J 

Z\T 


K^{t)Kr 



/ obi 


b2 


Kr 


bsj 



X 


(1 - \b3b^T~^Q~''''^^)\a\^"~"^^\bib2bs%\^^-^dt 

We consider the following change of variables. Set a tit'^^,bi 1 —!■ t 2 t 3 t 4 t 3 ,b 2 e- 
63 I—> and 64 I—> ts. With this change of variables the torus Z\T is parame¬ 

terized as t = diag{tit2t3t4t5,tit3t4t3,tit4t5,tit5,ti,t5,l,t^^ Thus 
the above integral equals 


C( 8 s - 2 ) f 

C( 8 s) J 

Z\T 


Kr{t)Kr 



t2 




Kr 


tl 


tb 


\ 

y 


X 
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(1 - 

For 1 < z < 5 write ti = p"'». Also, we set y = g- 2 s+i/ 2 ^ Denote by (mi, m 2 , m 3 , m 4 , ms) 
the trace of the irreducible representation ^ rriiWi evaluated at the semi-simple con- 
jugacy class of GS'pmio(C) associated with vr. Here wi is the i-th fundamental rep¬ 
resentation of GS'pinio(C). It follows from the Casselman-Shalika formula EISI that 
Kirit) = (n 2 , rea, n 4 , ns, m). Similarly, we denote by (m) the trace of the irreducible 
representation mw evaluated at the semi-simple conjugacy class of GL 2 {C) associated 
with r. Using again the Casselman-Shalika formula we obtain 


Kr 



t2 


( 






t5 


\ 

y 


I{n 2 ,n 3 ) (g) (ni) (g) (ns) 


where Z(n 2 ,n 3 ) denotes the restriction of the irreducible representation with highest 
weight [n 2 ,n 3 ] of GL 3 (C) to the group S 03 {C). 

Thus, to prove the proposition we are reduced to prove the identity 


^(n2, na, n4, ns, ni)fc(ni, n2, na, ns)2/"i+2n2+4n3+2n4+3n5(i _ yMn.+i)^ ^ 
rii 


(1 — L{SpiniQ x St, vr x r, 2 s — 1 / 2 ) 
where A:(ni, n 2 , na, ns) = /(n 2 ,n 3 ) (g) (ni) (g) (ns). For y as in section 3, we have 

L{SpiniQ X St, TT X r, 2s — 1/2) = L{Spinio, 0X, 2s- l/2)L(S'pinio, vr0x 2s- 1/2). 

By the Poincare identity, and Brion’s decomposition of the symmetric algebra of SpiniQ 
( see |Brj i we have 

00 

L(Spinio,7r (g) x, 2s - 1/2) = ^ (m, 0,0,0, 

m,£=0 


Let 

p(x, y) = (i-x®/)+(/2/®-x'2/')(i, 0,0,0,0)x'y"+X^/(0,0,0,1,0)-x"y®(0,0,0,0,1). 
Then, using again the Casselman-Shalika formula, we obtain 

00 00 

P{x,y) E Ko,o,o,^)x'-+y™+' = E(o,o,o,o,£)xV. 

m,,t=Q £=0 

So, our identity reduces to 

00 

P(X, y)P(x~\y) na, n 4 , ns, ni)A:(ni, n 2 , na, (1-/^”'*+^^) 

Tli 
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= (1 - /)2 ^ ( 0 , 0 , 0 , 0 , e) 0 ( 0 , 0 , 0 , 0 , m)x^-^y^+^. 

m,i=0 

Here and henceforth, by abuse of notation, we write (n2, ns, 714, ns, ni) for the repre¬ 
sentation itself, as well as the value of the trace at the conjugacy class associated to 

TT. 

Lemma 5: We have 

(0,0,0,0, £) (81 ( 0 , 0 , 0 , 0 , m) = (a, 0 , 5 , 0 , m^ — 2 a — 26 ). 

a+b<min(m,£) 

Proof: This result is due to Okada m see also m- ■ 

What remains is to check that 

00 

P{X, y)P{x~^,y) ns, n 4 , ns, ni)A:(ni, n2, ns, 

rii 


= I X] (n2,0,n4,0,ni-h£-2n2 - 2n4) I X™ (16) 

m /=0 \ri2-\-n4<rnin{rn,i) j 

= (1 - y^f Y. (^2, 0 , n 4 , 0 , ^^)^ 2 n.+ 2 n 4 +ni^-ni X _ 

1 — X 

n 2 ,n 4 ,ni =0 

We check (fT^ in two stages. First, we show that 

00 

r'(x,!/)j;(n2.n3,n4.n5,n,)t(ni,n2,n3.n5)!,”‘+"““+‘™+""‘+=‘"»(l-8"‘+‘') (17) 


= (1-!/VE 

rii 


\ _ ^ 2 (ni+l) ^ _ ^ 2 (n 2 +l) 

i-x' i-x^ 


(ni+ 2 n 2 + 2 n 3 +n 5 ) 


y 


ni+ 2 n 2 + 4 n 3 + 2 n 4 -|- 3 n 5 


Then, we check that P{x ^,y) times this sum is the desired sum. First, using the 
Casselman-Shalika formula, we note that A:(ni, n2, ns, ns) is equal to 


(1 _ y2(n2+l))(i _ ^2(n3+l))(i _ ^2(n2+n3+2))_ ^2(n4+l)^(i _ ^^(^5+1)) 

( 1 -X^)^(l-x"^) 

To check jm, we shall verify that the coefficient of (n2, ns, n4, ns, ni) is the same on 
both sides. Let 


9 lni] (X) 2 /) 


_ ^2(712+1)^^]^ _ ^2(713+1)^^]^ _ ^2(712+773+2)^^]^ _ ^2(711+1)^^]^ _ ^2(715+1)^ 

(i-x^)^(i-x'^) 


^-(771+2712+713+715)^711+2712+4713+2714+3715 _ ^4(714+1)1 
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Let Li denote the set of weights of ( 1 , 0 , 0 , 0 , 0 ), r4 those of ( 0 , 0 , 0 , 1 , 0 ), and Ls those 
of ( 0 , 0 , 0 , 0 , 1 ). Then, we claim that 

CXD OO 

P{x,y) '^{n2,n3,n4,n5,ni)g[ni]{x,y) = ns, 714 ,ns,ni)G(„,)(x,y) 

rii Tii 

where 

G{ni){x,y) = (1 -x®/)y[ni](x,y) + (x®y® -x^y^) 9 [ni-wi]ix,y)+ (is) 

liiSFi 


x^y^ X] 9 [ni-wi]{x,y) - x^y^ 9 ini-wi]{x,y))- 

w£r4 loeFs 

Indeed, Brauer proved in general that if xa denotes the character of the irreducible 
representation of highest weight A, then 

XAX/i ^ ^ ' 5 yX|A+y+p|—pj 


where the sum is over the weights of the representation with highest weight y, with 
multiplicity, |r| denotes the dominant member of the Weyl orbit of r, and Si, is equal 
to (— 1 )"' if tc(A + u + p) = |A + n + p| for a unique w, and zero if A + n + /? has a 
stabilizer in the Weyl group. See |Buj p. 171 and exercises. In our case, no element of 
riUr4Ur5 has an entry less than — 1 , so when A + n + p is not dominant, it has a 
stabilizer in the Weyl group. This proves m when no is zero. If any of the Uj is 
zero, there are extra terms on the right hand side corresponding io Ui — Wi = — 1 . But 
referring back to the formula for ,], we see that all these terms vanish anyway. 

Hence, to check (CZI), we now need to check that is equal to the coefficient 

of (n2, ns, n4, ns, ni) on the right side of (tTTj) . Replacing by W and y^"''* by 

Y4 we obtain an identity of polynomials in seven variables which is straightforward to 
verify by computer. 

Now, we need to check that 


Pix ^,y)Yl 

TLi 


1 _ ^ 2 (ni+l) ^ _ ^ 2 ( 712 + 1 ) 

1 - X^ 1 - x^ 


-(711+2712+2^3+715) ni+ 2 n 2 + 4 n 3 + 2 n 4 + 3 n 5 
A. y 


Let 


OO 

= (n 2 , 0 ,n 4 , 0 ,ni)y 2 - 2 + 2 n 4 + 7 ii^-ni 

712 , 71 - 4 , 711=0 


1 _ ^2(711+1) 

i-x' 


^K](x,y) 


1 _ ^ 2 (tii+ 1 ) _ ^ 2 ( 712 + 1 ) 

1 _ X 2 1-X^ 


^-(711+2712+2713+115) y Til +2712 +4713 +2714+3715 


when all n* are nonnegative. Note that this expression is equal to zero if ni or n2 is 
equal to — 1 . We extend the definition of to be zero if min (ns, n4, ns) = — 1 . Then 
the left hand side is equal to 


OO 

^(n 2 , ns, n4, ns, ni)iL(„,)(x, y), 

TLi 
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where 


^^K)(x,y) = (1-X ®/)/iK](x,y) + (x ®y®-x Y1 ^K-«;i](x,y)+ 

uiGPi 

x"^y^ X] h[ni-m]ix,y) - x~^y^ h[ni-wi]{x,y))- 

uiGr4 -iuers 

We claim that this is equal to 

1 _ 

2n2+2n4+ni ^-ni _ 

y X. 1 9 ’ 

1 -x^ 

when = 0, and zero otherwise. To show this, we partition each Tj into 

eight sets, according to whether the W 3 ,W 4 , and are positive or nonpositive. This 
determines in which the corresponding term is zero by convention. 

We replace ;Y 2 (ni+i) ^ 2 (n 2 +i) throughout. A factor of 

^-(ni+2n2+2n3+n5)yni+2n2+4n3+2n4+3n5 _ ^^^-2 


may be factored out of the sum and moved to the other side, which then becomes 

i?(x,Xi,A2) = ^(l-Ai)(l-x'), 

when na = ns = 0. On the other side of the equation we have a polynomial obtained 
by summing 

“ j^^^—2w2^^wi+2w2+2w3+W5y—(wi+2w2+4:W3+2w4+3w5) 


over the weights, with the correct coefficients from P. We break this into eight pieces. 

Let Qnnn denote the sum over weights where W 3 ,W 4 ,W 5 are all nonpositive, Qppp 
the sum where they are all positive, Qnpn the sum over weights such that W 3 and W 5 
are nonpositive, and W 4 is positive, and so on. Since (1 — X*y*)^[ni] (X) y) is there for 
every [m], it is included in Qnnn- The answers are: 


Qppp Qppn Qnpp Qnpn 0 

Qnnn — Qpnp — R 
Qnnp — Qpnn — R- 


From here it is an easy application of the inclusion-exclusion principle. (It is easy to 
check by hand that Qppp = Qppn = Qnpp = 0, since in all three cases the set of roots 
satisfying the desired condition is empty.) 
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5 A construction of a certain lifting 

In this section we shall use a certain small representation, defined on the group 5022(A), 
in order to construct the endoscopic lifting from PGL 2 x SpQ to the group 50io. Then, 
in the next section we will use this construction to prove our main result. 

This construction is a special case of a more general set of constructions which are 
defined on classical groups. It is a part of a work in progress of the first named author 
and was partly announced in |G2j . To make things clear, we shall now sketch this 
construction and just state the main results. The details of the proofs will appear in 

[nij. 

Let r denote a cuspidal representation of PGL 2 {A). In section 3 we constructed 
a small representation 9r, defined on the group GSOio{A). Clearly the construction 
there is also valid if we consider the group 50io instead. Recall that cr(r) is the cuspidal 
representation of GL 3 (A) obtained by the symmetric square lift of r, as constructed 
in |G-.Ij . Let ET{g,s) denote the Eisenstein series defined on GLq{A) associated with 
the induced representation Ind^^^^\a{T)®a{T))5'\^. Here L is the maximal parabolic 
subgroup of GLq whose Levi part is GL 3 x GL 3 . It is well known, see for example [j], 
that this representation has a unique simple pole and its residue, which we shall denote 
by Er, is the well known Speh representation. Let Q denote the maximal parabolic 
subgroup of the split orthogonal group SO 22 , whose Levi part is GLq x 50io. Let 
Er{m, s) denote the Eisenstein series defined on 5022 (A) associated with the induced 
representation Ind^^j'^\Er G dr)dQ. 

The following two facts about this Eisenstein series are proved in |G3j . 

a) In the domain Re{s) > 1/2, the Eisenstein series Er{m,s) has a unique simple 
pole. We shall denote the residue representation which is obtained by Or- By abuse of 
notations we shall denote by 9r{m) a vector in this representation when realized in the 
space of automorphic forms. 

b) Using the notations introduced in the beginning of section 2, we have 05022 (^ 1 -) = 
(3^1). This means that Or has a nonzero Fourier coefficient which corresponds to the 
unipotent orbit (3^1). Also, for any unipotent orbit O which is greater than or not 
related to (3^1), the representation Or has no nonzero Fourier coefficient corresponding 
to the orbit O. 

At this point we are ready to introduce the global lifting we need for our result. Let 
ei denote a cuspidal representation defined on the group Spq{A). Let 0^ denote the 
theta representation defined on the group 5pgo(A). Here Sp denotes the double cover of 
the symplectic group and 4> denotes a Schwartz function defined on Let U denote 
the unipotent radical subgroup of Q and let Llgi denote the Heisenberg group with 61 
variables. We shall write an element h G iLgi as h = (X,Y,z) where X,Y € Mat^xio 
and 2 ; G Matixi- We define a homomorphism I : U ^ LI 31 as follows. In terms of 
matrices we can identify U with the group of all matrices 



(h Y Ri R 2 \ 



h X R 3 Rl 


u = 

ho X* Y* 



h 



\ h) 
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Here X,Y E Mat^xio^ Ri ^ and R2,R3 S -^®^3x3 ~ ^ Mat^xs ■ 

R^Js + J3R = 0 }. The matrix J3 was defined in the beginning of section 2 . For all 
u € U as above, if Ri = {vij) we define l(u} = (X,Y,ri^i + r2,2 + ^3,3)- This map is 
clearly onto. 

We now consider 

fi{g)= j J ipei{h)9^{l{u){h,g))6r{u{h,g))dudh ( 20 ) 

Spe{F)\Spe{A)U(F)\UiA) 

Here g E 5 'Oio(A), and the embedding of the groups SpQ and SOio in Speo and SO22 
are as follows. First, inside Spqq we embed these two groups via the tensor product 
representation. In SO22, we embed them in the obvious way inside the Levi part of Q 
which is GLq x SOiq. 

Let TTi denote the automorphic representation on S'Oio(A) defined by all right 
translations of the functions fi{g) defined in (I 2 ()jl . We summarize some of the properties 
of this representation. The proofs are given in [(I. 3 | . 

1 ) The representation tti is a cuspidal representation provided a certain integral is 
zero. This is the tower property, and the certain integral we refer to is the lifting in the 
previous stage. Since we will not need this point we shall not discuss it any further. 

2) The representation tti is generic if and only if ei is a generic cuspidal representation 
of Spe{A). 

It is important to us that we can interchange the representations vri and ei in 
integral (PH) . More precisely, let now tt denote a cuspidal representation defined on 
S'Oio(A). Then we can form the space of automorphic functions defined by 

f{h)= J J (p^{g)e^{l{u){h,g))er{u{h,g))dudg ( 21 ) 

SOio{F)\SOio(A) U{F)\U{A) 

Let e denote the automorphic representation of Spq{A) defined by right translations 
of the above functions f{h). The last fact that we need, and which will be proved in 
m, is the relations at the unramified components of each of these representations. 
More precisely, let vr, e and r be as above. Suppose that the integral 

fe{h)p^{g)e^{l{u){h,g))9r{u{h,g))dudgdh (22) 

SpeiF)\Sp6{A) SOw{F)\SOio(A) f/(F)\[/{A) 

is not zero for some choice of data. Then we have a ’’Howe duality” type of result. 
That is, 

3) Suppose that integral (1221) is not zero for some choice of data. Let be a finite 
nonarchimedean place where all the data is unramified. Let 9r^u, and denote 
the local representations corresponding to the global ones which appear in integral 
(HH). Then, that integral induces an element in the space 

HoTflgpgxSOio ® ® Fy') 

where (...)[/ is the corresponding Jacquet module. As in 1021 section 6, it will be 
proved in |(I 3 j . that any two of the representations Ty,ey,Fy determines the third one 
uniquely. 
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From this we deduce that the representation tti, as defined by integral CT . if 
nonzero, is the weak functorial lift from ei and r. Similarly, if nonzero, integral m 
defines an automorphic representation e on Sp^^A), such that the representation vr is 
the weak lift from e and r. In the following Lemma, we will prove that e, as defined by 
integral m is cuspidal. We don’t know if the image of the lift is irreducible, however, 
it is clear that all irreducible summands are nearly equivalent. 

We start by proving 

Lemma 6: The representation e is cuspidal. 

Proof: Let 1^ be a standard unipotent subgroup of SpQ. By standard we mean that V 

Js'' 


consists of upper unipotent matrices. We shall use the form 
the symplectic group SpQ in terms of matrices. Write 


-Js 


to represent 


9^{l{u){v,g)) = ^ ^ uj^{{X+C,Y, z){v, g))(j){0) 

^£Mat3xlo{F) i&Mat3xlo{F) 

Here is the Weil representation and the above equalities are obtained from the well 
known formulas for the Weil representation ( see for example |(T-R.-S3j i. Plugging this 
identity into m, collapsing the summation and the integration over the X variable 
in U, we deduce that the integral f f{v)dv is zero for all choice of data, if and 

V{F)\V(A) 

only if the integral 


/ / 


PTT{g)Or{ui{v,g))i)i{Ri)duidvdg (23) 


50io(F)\SOio(A) V{F)\V{A) Ui{F)\Ui{A) 


is zero for all choice of data. Here Ui is the subgroup of U which consists of all matrices 
as in (O such that X = 0. The character 'i/^i is defined as follows. Write ui in the 
coordinates as given in (Hill) . Ifiii = (vij) G Mataxs, then='^/’(^l,l+^ 2 , 2 +^’ 3 , 3 )■ 
At this point we need to consider the three nonconjugate maximal unipotent radicals 
of SpQ. We shall work out the details in the case where V is the unipotent radical of 
the Siegel parabolic. The other two cases are treated similarly. Let 


V = 





Z G Mat3x3',Z J 3 — J 3 Z 


Thus (Oni) is equal to 



/h 

z 

Y Ri 

R2\ 



h 

R 3 

Rl 




ho 





h 



V 



h) 



il!i{Ri)duidZdg 


(24) 


Here g and ui are integrated as in (|^ and Z is integrated over V{F)\V{A). If 
Z = (zij) we consider the Fourier expansion of the above integral along the variables 
-^ 2 , 3 , -^ 3,2 and 2 : 3^3 each integrated over F\A. Conjugating by a suitable discrete matrix 
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in i ?3 we collapse summation and integration, and deduce that the vanishing of (HI 
for all choice of data is equivalent to the vanishing of 


/ 




//3 

Z Y 

Ri 

R2\ 


I 3 


Rl 


ho 





h 

z* 

V 



h) 




ijji{Ri)duidZdg 


(25) 


for all choice of data. Now Z is integrated over Mat 3 x 3 (i^)\Mat 3 x 3 (A). Let U 2 denote 
the unipotent subgroup of SO 22 which consists of all matrices of the form 



(h z Y Rl R 2 \ 



h Rl 


u = 

ho Y* 



I 3 z* 


K 

\ hj 

> 


(26) 


Conjugating integral (1^ by a suitable Weyl element, it is then enough to show 
that the integral 




(h Rl Y Z R 2 \ 


f ~ 


I3 z* 


/ ‘fiTT{9)dT 


ho Y* 

h,9) 

1 


h Rl 




\ h) 



ipi{Ri)du2dg 


(27) 


is zero for all choice of data. Here U 2 is integrated over t/ 2 (H)\t/ 2 (A). Next we consider 
the Fourier expansion of along the unipotent group 


(h \ 

h R3 
ho 

h 

\ h) 


R 3 S Mat3x3 


with points in F\A. Under the action of GL'i{F) embedded as diag{m, m, Iio, m*, m*), 
there are two orbits to consider in this expansion. The first one is given by a sum of 
integrals of the form 


/ 





(h Rl Y Z R 2 \ 



h R 3 z* 



ho Y* 

( 1 , 5 ) 


h Rl 



\ h) 



hi{Ri)'^2{R3)du2dR^dg 


(28) 


where ip 2 {R 3 ) = hhi, 2 ) for all i ?3 = After some further tedious expansions 

and suitable conjugations one can show that the above integral is a sum of Fourier 
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coefficients where each one of them corresponds to a unipotent orbit which is bigger 
than or equal to (41^®). Since Oso 22 {^t) = (3^1) it follows that (1281) is zero for all 
choice of data. Thus we are left with the trivial orbit. That is, we need to show that 
for all choice of data the integral 


J ¥^7r(5)^r 


(h Ri 

V z 

R 2 \ 



h 

R3 

Z* 




ho 


(i>ff) 

ijji{Ri)du 2 dR 3 dg 


I3 

Rl 



V 


h) 




is zero. Next we expand integral dH along the unipotent set 


(29) 


/h 

V 


h 


X 

ho X* 

h 


\ 



X G Matsxio 


with points in F\A. The group GL'i{F) x SOiq{F) acts on this expansion with various 
orbits. The orbits can be identified with vectors G F^^, for 1 < i < 3, according to 
the length of these vectors. If at least one of these vectors has nonzero length, then 
the corresponding Fourier coefficient contains as inner integration a Fourier coefficient 
which corresponds to the unipotent orbit (51^^). By the smallness of 9r this Fourier 
coefficient is zero. Thus we are left with the cases where the length of all vectors is 
zero. Hence we can write the above Fourier expansion as a sum of integrals of the form 



(h Ri 

Y 

Z 

R 2 \ 


h 

X 

R3 

z* 



dio 

X* 


( 1 , 5 ) 



h 

Rl 


V 



h) 



ipi{Ri)'ipi2{X)dXdu2dR^dg 


(30) 


Here u = (z^i, 1 ^ 2 , t's) is one of the vectors (0,0, 0), (1, 0,0), (1,1,0) or (1,1,1). The 
characters hu are defined as follows. Given X = (xij) G Mat^xio we define hu{X) = 
+ ^ 2 X 2,2 + ^ 3 X 3 ^ 3 ) where u is any one of the above four vectors. The vari¬ 
able g is integrated over SOiq_ 2 (ui+u 2 +u 3 ){R)Uu\SOio{A), where Uu is the unipotent 
radical of the maximal parabolic subgroup of S'Oio whose Levi^part is GLix^+i, 2 +u 3 x 
S'Oio_ 2 (^i+t/ 2 +i/ 3 )- If = (0,0,0) then from the definition of Or we obtain the inte¬ 
gral f ^■w{g)0T{g)dg as inner integration. Here g is integrated over S'Oio(T)\S'Oio(A). 
Hence by the cuspidality of vr this integral is zero. If v is any one of the other three 
cases, we continue by expanding integral (EDI) along the group Uy{F)\Uu{A). If this 
group is not abelian, we first expand along its center. By the smallness of Or we ob¬ 
tain that all orbits in the various expansions are zero except the constant term along 
Ui,. This follows from the fact that each nontrivial Fourier coefficient will contain, 
as inner integration, a Fourier coefficient corresponding to a unipotent orbit which is 
greater than or not related to (3^1). Hence it will give zero contribution. The constant 
term will also vanish. Indeed, in this case after factoring the integral, we will obtain 
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the integral f ip.„-{ug)du as inner integration. Here, the variable u is integrated over 
Uy{F)\Ujj{A.). Hence we get zero by the cuspidality of tt. This completes the proof of 
the vanishing for the unipotent radical of the Siegel parabolic. We still have to prove 
the same for the other two maximal unipotent radicals. This is done in a similar way 
and we shall omit the details. ■ 


6 The Main Theorem 

We recall the basic notations we used in the previous sections. Let tt denote a generic 
cuspidal representation of GSOio{A). We shall assume that tt has a trivial central 
character. Let r denote a cuspidal representation of PGL 2 {A). In section two we 
introduced the global integral 


^7T{9)0T{9)E{g,s)dg (31) 

Z(A)G(F)\G{A) 

Here G = GSOio, and Z is the center of G. The representation 6r was introduced 
in section three, and the Eisenstein series E{g,s) was dehned at the beginning of 
section two. It follows from sections 2 and 3 that m unfolds to a Whittaker integral, 
and hence is Eulerian. From Proposition 4 in section 4 we deduce that at the non- 
archimedean unramified places this integral represents the L function L{Spinio ^ St, tt x 
r, 2s —1/2). Let S denote a finite number of places in F which includes all archimedean 
places, such that outside of S all data is unramified. We shall denote by L^{Spinio x 
St, TT X t,2s — 1/2) the corresponding partial L function. 

It follows from EEl that the Eisenstein series E{g,s) can have at most a simple 
pole at the points s = 1 and s = 3/4. The residue at s = 1 of E{g,s) is the trivial 
representation, and hence the residue of integral m is zero at that point. We are 
interested in the residue of m at the other point. In this section only, we shall denote 
by 9 the residual representation of E{g,s) at s = 3/4. In other words, we denote 
9{g) = ReSg^'^i^E{g,s). It follows from |(I-B,-S4j section 6 that 6 is the irreducible 
minimal representation of G. We are now ready to state and prove our main result 
Main Theorem: Let tt be an irreducible generic cuspidal representation of the group 
G{A) which has a trivial central character. Then the following are equivalent: 

1) The partial L function L^{Spinio x St, tt xt,2s — 1/2) has a simple pole at s = 3/4. 

2) The period integral 


Tni9)9r{9)9{g)dg (32) 

SOio(F)\50io(A) 

is nonzero for some choice of data. 

3) There exists a generic cuspidal representation a of the exceptional group 6*2 (A) such 
that TT is the weak lift from the representation a x t of the group G 2 (A) x PGL 2 {A). 
Proof: Two parts are not hard to prove. Suppose 1) holds. Arguing as in usi section 
7, one can show that for any place in the global field F, given a value of s G C, one 
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can choose data such that the local integral m is nonzero. From the results in the 
previous sections we hence deduce that the integral 

y^7r{9)6T{g)0{g)dg (33) 

Z(A)G50io(F)\G50io(A) 

is nonzero for some choice of data. Factoring the similitude in dSSl), we deduce that if 
the partial L has a pole at s = 3/4, then integral (1321) is nonzero for some choice of 
data. Thus 1) implies 2). 

The implication 3) implies 1) is also not hard. Suppose that vr is the weak lift from 
a XT. Then branching from S'Oio(C) to G 2 (C) x 5^2(0) we obtain that L^{Spinio ^ 
St, TT X t,2s — 1/2) is equal to 

L^{G 2 X Sym^, a x t,2s - 1/2)L^(G2, a, 2s - 1/2)L^{Sym^ ,t, 2s - l/2)C^(2s - 1/2) 

Here G 2 is the standard seven dimensional representation of G 2 (C) and Sym? is the 
symmetric square representation of SL 2 {C). Also, C^{2s — ll2) denotes the partial 
global Q function. Since a is generic it follows from |(l-R.-S5j that a has a nontrivial 
lift to a generic cuspidal representation of Spq{A) or to a cuspidal representation on 
PGL^{A). Using the lifting of generic cuspidal representations from Classical groups 
to GLn as proved in |0-K-PS-S| . we deduce that the above product of L functions is 
actually a product of partial L functions associated with tensor product representations 
on certain GL’s. Hence, at s = 3/4 these L functions cannot vanish. On the other 
hand, C^{2s — 1/2) has a simple pole at that point. Hence 3) implies 1). 

Finally we prove that 2) implies 3). Before providing the details, let us first explain 
the idea of the proof. Assuming (jSl is nonzero for some choice of data, we need to 
construct a generic cuspidal representation a of the group 0*2 (A) and prove that vr is 
the weak lift of u x r. Suppose that there exists a cuspidal representation e of Spq{A) 
such that the integral 


= J j f{vm)^v{v)dvdm (34) 

SL2{F)\SL2(A)V{F)\V{A) 

is nonzero for some choice of data. Here / is a vector in the space of e and the group 
R = V ■ SL 2 is defined as follows. Let V denote the unipotent radical of the maximal 
parabolic subgroup of SpQ whose levi part is GL 2 x SL 2 . Thus in matrices we have 


V-512 


(I 2 X 

( 




: X G Mat 2 x 2 ',y G Af0^2x2!"^ ^ 


rrij 


Here Mat 2 x 2 — {v ^ Mat 2 x 2 ■ y^J 2 = <^ 22 /}- Also, we define tpviv) = ipitrx). It is 
also not hard to prove that if e is such that integral (El) is nonzero for some choice of 
data, then e must be ereneric. Assumine all this, it follows from |(T-Ja| ( see also iniHI 
the discussion before and after equation (27)) that e is the weak functorial lift from a 
cuspidal representation a on G 2 (A). 
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Let e denote the representation defined by right translations of all functions given 
by integral m- As proved in section 5 we know that e is a cuspidal representation of 
Spq{A). Thus if we can prove that is nonzero for some choice of function f{h) 

as in m it will follow from section 5 that vr is a weak endoscopic lift from ex t. From 
the above discussion it will follow that e is a lift from a generic cuspidal representation 
a of G 2 (A). This will prove that 2) implies 3). 

Thus, to conclude the proof of the theorem we will show that if (El is not zero for 
some choice of data, then the integral 


V^TT { 9 )S g))9r{u{vm, g))'il)v{v)dudgdvdm 

SL2(F)\SL2{A)V(F)\V(A) U{F)\U{A) 

(35) 

is not zero for some choice of data. Here g is integrated over 50io(T)\S'Oio(A). Since 
the inner integrations, over the variables u and g produce a cusp form on Spq{A), the 
integral over SL 2 {F)\SL 2 {A) converges. At some point we will need to interchange the 
order of the integration. The justification for this is given in EH Proposition 5.3.1 . 
In other words, one needs to add the action of the Casimir element for SL 2 inside the 
Schwartz function inside the theta representation. Doing that, the theta representation 
becomes rapidly decreasing as a function of the SL 2 . Because of |(I-R.-S4) Theorem 
6.9 this will not harm the generality of our argument. 

We start by unfolding the theta series. As in section 5 we have 

6^{l{u){vm, g)) ='^uj^{l{u){vm,g))(l){g,^) = '^uj^{l{g)l{u){vm,g))(t){Q,i). 

Here g € Mat 2 xio and ^ ^ . Plugging this into (1351) we collapse the summation over 

g with the suitable integration inside U. We then consider the Fourier expansion along 
the unipotent subgroup of SO 22 defined by I 22 + 3 + r 2 ej 4 + r 2 e 2 3 + r 2 e 2 4 + g- 

Here e[ j = Sij — e 23 -j, 23 -i and eij is the matrix which has one at the (z,j) position 
and zero elsewhere. Once again, collapsing summation and integration, integral ( 1 ^ 
equals 

/ oj^{l{u 2 ){m, g)l{ui))(j){0, ^)9riu3U2im, g)ul)^l;u3iu3)di■■.) (36) 

Here, the variables m and g are integrated as in El- In term of matrices we have 



/h 


Ul = 


h 


h 


Zi 

Z 3 Z 2 

h3 Zl 

h 


h 


(h 


U2 = 


I 2 J 


I2 yi 

h 

Iio 


h 


2/2 

vl 

h 


(37) 


I 2 / 


Here, the variable zi is integrated over Mat 2 x 2 {A), the variable Z 2 is integrated over 
-^n^ 2 x 2 (''^) ( see the definition right after (HI) and Z 3 is integrated over Mat 2 x 10 (A). 


23 










The variable yi is integrated over Mat2xio{F)\Mat2xioi-^) and 1/2 is integrated over 
Mat2x2(-^)\-^®^2x2(-^)- variable U3 is defined as 


/h 


U 3 = U3{xi,X2,X3,ri,r2,rs,r4) = 


Xi X2 

h X 3 

h 


n r2 rs rA 


'10 


V 


( 38 ) 


hj 


Here all variables are integrated over Mataxb, for suitable choice of a and b with 
points in F\A, except r4 which is integrated over Mat^x2- Finally, the character 
is defined as follows. For r2 = (rjj) and X3 = (xjj) as described in (i 38 |) we set 
= ' 0 (^ 1 ,! + X2,2 + 3 ^ 1 ,1 + 3:2,2)- Notice that the variable v which appeared inside 
the Weil representation one equation before (l 36 |) does not appear inside u).^ in (l 36 |) . 
This follows from the fact that 1 ))(^( 0 ,^) = (^( 0 ,^). 

The group 1x3(0, r4) is the one dimensional unipotent group which is the center 
of the maximal unipotent subgroup of S022- Let N2 denote the standard unipotent 
radical of the maximal parabolic subgroup of SO22 whose levi part is GL2 x S’Ois- We 
have 

j 0^(113(0, r4))(ir4 = ^ j er{n 2 )'il^{a ■ n 2 )dn 2 

F\A " N2(F)\N2iA) 

where the sum is over all a € Mat2xi3{F). The group SOi%{F) acts on this expansion. 


If a = 


Cll 

0 L 2 


where ai G F^^, then the various orbits can be parameterized by the rank 


of the matrix a and by the length of ctj. 

Plugging this expansion into we claim that only one orbit contributes a nonzero 
term. Indeed, after plugging in (EEI), we conjugate the matrix it3(xi,X2,0,ri,r2,rs,0) 
from right to left. Because of the character we get zero contribution unless the 
rank of a is two and both vectors ctj are nonzero and have zero length. From this we 
deduce that equals 

[ ^nig) ^ uj^{l{u2){m, g)l{ui))(f){ 0 ,0 X] G!^^’'^{'yusU2{m, c/)ui)V'f/3 Mdi-) ( 39 ) 

« 7 

Here 7 G P2{SOi4){F)\SOis{F) where P^iSOu) is the subgroup of SOis which con¬ 
sists of all matrices of the form 



Also, we denote 


6r{n2k)ipN2{n2)dn2 k € SO22 


N2{F)\N2{A) 
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where is defined as follows. Identifying N 2 with the group of all matrices of the 
form 77,2 = U 3 (a;i,a; 2 , 0 ,ri,r 2 ,r 3 ,r 4 ), we set 'ijjN 2 {n 2 ) = 'ip{trxi). 

Next we consider the double coset space P 2 {SOii)\SOi%/P 2 {SOi 4 P). Here P 2 {SOi 4 ) 
is the standard parabolic subgroup of 50i8 whose levi part is GL 2 x SOu. Thus 
P 2 {SOi 4 ) contains P^iSOn). Considering the various orbits and arguing as above, one 
can show that only one orbit, which we shall describe bellow, contributes a nonzero 
term to (Ii-i9|) . Let 


Wi 


(h \ 

h 
h 

ho 

h 

h 

\ hj 


m 


(h \ 

h 5 
h 

ho 

h 

h 

\ h J 


where 5 € Mat 2 x 2 - Then integral dsni) equals 


C 7,-5 

(40) 

Here 7 € P 2 {SOio){F)\SOi 4 ^{F) where P®(•S'Oio) is the subgroup of S'Oi 4 which con¬ 
sists of all matrices in of the form 

( I 2 * * \ 

A: * j k€ SOio 


Also, we have S G Mat 2 x 2 {F)- In 1^01) we conjugate the matrix n3(xi,0) across t{5). 
We obtain from the commutation relations the matrix 773 ( 0 , xi5, 0). When we conjugate 
this matrix across wi and change variables in N 2 , we obtain the integral 

J 'ip{tr{xi5))dxi 

(nA)4 


as inner integration. Clearly this integral is zero unless 5 = 0. Thus, if 5 7 ^ 0 in (001) 
we get zero contribution. 

Next we consider the space P^iSOiohSOn/P 2 {SOio). Here P 2 (<S'Oio) is the stan¬ 
dard parabolic subgroup of SOu whose levi part is GL 2 x SOiq. Thus P 2 {SOio) 
contains PP^iSOio). As before one can check that all orbits contribute zero except one. 
Let 


W2 



h 


h 


ho 


h 


\ 


h 


\ 


hi 




(h \ 

h 

I 2 5 
ho 

h 

h 

\ hi 
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where 6 G Mat 2 xio aiid fj. G Mat2x2- Conjugating by the matrices 7/3(0,X2,0,ri,0) 
across from left to right, as we did before with U 3 {xi, 0 ), we obtain that only when 
(5 = 0 and fi = 0 then we get a nonzero contribution. Thus, integral (lini) equals 


(41) 


where we wrote U3(x3) for 7/3(0 ,0,X 3 ,0). 
Next, we consider the Fourier expansion 



One can check that if /3 7^ 0 then the corresponding summand is zero. Indeed, in this 
case we obtain as inner integration, Fourier coefficients of Or corresponding to unipotent 
orbits which are greater or equal to (41^®). By section 5 point b) it follows that all 
these Fourier coefficients are zero. Thus we are left with the summand corresponding 
to /3 = 0. This we can further expand along the set 


(h \ 

I2 X * 
lu X* 

h 

\ h J 


where x G Mat 2 xi 2 - The group SO 12 acts on this expansion, and as before we can 
parameterize the various orbits by matrices a = f ) G Mat 2 xi 2 {F) where the in- 

V“2/ 

variants are the rank of a and the length of Oj. Once again we are left with the 
contribution of one orbit which corresponds to rank two matrices such that both Ofj 
are nonzero vectors of length zero. Thus, integral m is equal to 


/ ^Trig) ^ uj^il{u 2 ){m, g)l{ui))(p{0, ^ 6»f'‘’’^(77Ci7n27/3(x3)u2(m, g)ui)'il>{trx 3 )d{...) 

(42) 


Here, the group N4 consists of all unipotent matrices in SO 22 of the form 


and 


/I2 ri * * * 

I 2 r2 * * 

h 

ho 

h 

V 


* * \ 

* * 

* * 


* * 
rl * 
h rt 

h / 




Or {n4,k)%ljN4, {ni ) dui 


Ni{F)\N4{A) 
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where V’iV4(^4) = ' 4 ’itr{ri +r2)). Also, we have 7 G P2 {SOio){F)\SOi4{F). Continuing 
this process of Fourier expansions it follows from the smallness properties of 9r that 
= 6r^’^{k) where 

0N6F(^k) = j 9r{nek)'tljNeine)dnG 

Ne{F)\Ne{A) 

Here Nq is the unipotent subgroup of SO22 which consists of all matrices of the form 

/h n * * * * *\ 

I2 r2 * * * * 

-^2 ^3 * * * 

ho * * 

h * 
h rt 

\ h J 

and V'Afe is the character extended trivially from N4 to N^. 

As before we consider the space F2{SOiq)\SOi4/F2{SOiq). We obtain a nonzero 
contribution from one term. Thus, integral (EH) equals 


uj^{l{u2){m, g)l{ui))(j){ 0 ,0 ^ 9^^’^{w2t{6, g)wiW2U2{m, g)ui)d{...) ( 43 ) 




Conjugating W2Wit{6, fi)wiW2 we see that as a group, this group of matrices coincides 
with the group U2- Thus we may collapse summation with integration. Hence ( 1151 ) is 
equal to 

M 9 )Y 1 g)l{u2Ui))4’{0, {w2WiW2{m, g)u2Ui)duidu2dgdm 

{UiU2){A) 5 

( 44 ) 

Here the variables m and g are integrated as before. That is, the variable m is integrated 
over SL2{F)\SL2{A), and g is integrated over SOio{F)\SOio{A). 

Notice that Nq = UVq where U is the unipotent radical of the parabolic subgroup 
Q as defined in (fT 9 |). The group Vg is the group of all unipotent matrices of the form 



V(i 


(h 


ri r2 
I2 rs 

h 


'10 


h 


\ 


'1 

h) ] 


To complete the proof of the theorem, we assume that part 2 ) in the statement of 
the Theorem holds, and assume that integral dm) is zero for all choice of data. We shall 
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derive a contradiction. First, using arguments similar as to those in Ida-Sj section 7, 
we may deduce that if indeed (EH) is zero for all choice of data, then so is the integral 

¥^vr(5') ^ {w 2 WiW 2 {m, g))dgdm 

SL2iF)\SL2{A) SOio{F)\SOio{A) ^ 

(45) 

Here we wrote 4> = (pi ^ (p 2 where (pi is a Schwartz function of Mat 2 xio(A) and (p 2 is 
a Schwartz function of Since W 2 W 1 W 2 normalizes the group of matrices {m,g) as 
above, we conjugate it to the right and may ignore it. 

Next, we claim that 9r^’^{{m,g)) = {{I, g)) for all m G SL 2 {A) embedded in 

S 022 {A) as in (1151) . We will prove it in Lemma 7 below. Assuming that, we deduce 
that if EH is zero for all choice of data, then so is the integral 

Mg)fig)o?^^^{ihg))dg (46) 

SOio(F)\SOio{A) 



where we denoted 

fid) = j ‘^(piim, g))(p2{0dm 

SL2{F)\SL2{A) ^ 

Notice that the inner summation is in fact the theta representation defined on the 
group S'p 2 o(-^)' As explained in the beginning of the proof, after possible adjustments 
of the Schwartz function, it follows from Theorem 6.9, that f{g) is actually 

the minimal representation of SOm which we denoted by 9. Hence, from (1461) and the 
above discussion, we may assume that the integral 

^n{g)9{g)9^^’'^{{l,g))dg (47) 

SOioiF)\SOio{A) 

is zero for all choice of data. Recall that Nq = UVq. Also the character 'ip^g is trivial 
on Uq. In Lemma 7 below, we shall prove that 

J Er{vh)'ipVe{v)dv (48) 

Vs (F) We (A) 

is nonzero for some choice of data. Here Er is the Speh representation as was defined 
at the beginning of section 5, and by abuse of notations, we view Vg as a subgroup of 
GLq. The character V’vfe is the restriction of V’A 4 to the group Vg- 

As a function of g & SOio{A), the constant term 9^{{l,g)), is realized in the 
representation space of 9r. In other words, in view of the nonvanishing property of 
(1151) . it follows that the vanishing for all data of EH implies that El is zero for 
all choice of data. But this contradicts our assumption. Thus 2) implies 3). This 
completes the proof of the main theorem. ■ 
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We still need to prove 

Lemma 7: Let Ej- denote the Speh representation on GLq{A) as defined at the 
beginning of section 5. Then the integral 

T{h) = j Er 

(Mat2x2(F)\Mat2x2(A))3 



'il){tr{x + y))dxdydz (49) 


is nonzero for some choice of data. Moreover, ifh = diag{m,m,m), wherem G SL 2 {A), 
then the E{h) = E{e). 

Proof: It follows from [(44j Proposition 5.3 that 0{Er) = (3^). This means that Er 
has no nonzero Fourier coefficients for any unipotent orbit which is bigger than or not 


related to (3^). Let t{r) = diag{ 


1 


1 


1 


We expand E{h) along 


the group of matrices t{r) with points in F\A. We have 


E{h) = ^ f E{t{r)h)'il>{ar)dr 


If a is not zero then the corresponding expansion consists of Fourier coefficients for 
Er which correspond to unipotent orbits which are greater than or not related to (3^). 
Hence, if a 7 ^ 0, we get zero contribution which means that E{t{r)h) = E{h) for all 


r G A. Let v 


. Denote w = diag{v, v, v). Then clearly E{wh) = E{h). Since 


w and t{r) generate SL 2 {A), the second assertion follows. 

As for the first part, we shall assume that E{h) is zero for all choice of data and 
derive a contradiction. Let w denote the Weyl element of GLq with one at positions 
(1,1); (2, 3); (3, 5); (4, 2); (5,4); ( 6 , 6 ) and zero elsewhere. Conjugating w from left to 
right we deduce that the integral 



{F\A )12 


/ 

/i Xi X2 n r2\ 





1 X3 r3 


1 



1 


1 



1 Vi 2/2 


Zl Z2 1 



1 2/2 


Z3 1 



VV 1/ V 1// 


V’(xi+X3+yi+?/3)d(...) 


(50) 

is zero for all choice of data. Expand ((Sni) along the unipotent group consisting of all 
matrices d6 + X 4 ei ^4 + r 5 e 2,5 + r 6 e 3^6 where G F\A. Conjugating by a suitable discrete 
matrix and collapsing summations with integrations, dsni) is equal to 


/ 

/I Xl 

X2 

X4 

n 

X2\ 


1 

X3 


X5 

X3 



1 



re 




1 

2/1 

2/2 





1 

2/2 



VV 1/ 


/I 


V 


1 


Zi 


\\ 


1 

^2 1 
23 


ipixi +X3 + yi + y3)d{...) 


1 


1 // 


(51) 
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where the variables Zi are integrated over A and all the rest over F\A. Thus we 
conclude that integral m is zero for all choice of data. As in Ida-Sj section 7, it 
follows that we may ignore the integration over the Zi variables. That is, the inner 
integration is zero for all choice of data. Expand the inner integration in m along 
Iq + ^ 762,4 + rges^s where rj G F\A. We claim that except the constant term, all other 
terms contribute zero to the expansion. Indeed, this follows from the fact that each 
other term produces a Fourier coefficient for Ej- which corresponds to a unipotent orbit 
which is greater than or not related to (3^). Next, expanding along I + r 9 e 3^4 with 
rg G F\A, all terms, except the constant term, contribute zero. Thus we may deduce 
that the integral 



/ 

/I Xi 

X2 

r4 

ri 

r2\ 

\ 



1 

X3 

rj 

r5 

rg 


/ Er 



1 

rg 

rg 

xq 


J 




1 

Vi 

2/2 


(F\A)12 





1 

2/2 



V 

V 




1/ 

/ 


ij{xi + X3 + yi + ?/3)d(...) 


(52) 


is zero for all choice of data. However, from the definition of the Eisenstein series 
Frig, s), see beginning of section 5, it follows that this last integral cannot be zero for 
all choice of data. Thus we obtained a contradiction. ■ 
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